User Experience Evaluation of IT Support Mobile Application Using System Usability Scale (SUS) and Retrospective Think Aloud (RTA)

The development of today's digital world has been running more rapidly at the time of the presence of covid-19 pandemics, in which all aspects of life are very dependent on digitalization that time, and companies keep doing the work of enhancing innovation in their businesses, which is the developing of IT such as mobile application. PT XYZ is one of the companies engaged in printing and digital services. PT XYZ uses the mobile application IT Support to assist technicians in completing reports of printer damage experienced by consumers. During the order services process, users oftenly feel that menu on mobile application IT Support not effective and efficient yet on helping them to process order service, the elements in the mobile application are inconsistent so it complicated the user. To improve and finding the usability problem specifically needed the process user experience evaluation mobile application with system usability scale (SUS) and retrospective think aloud (RTA). Total sample used in the research was 52 respondents from active user IT Support using purposive sampling technique. The SUS score result is 50.77 in grade D. The less satisfied, because the SUS score is still smaller than 68. Retrospective Think Aloud (RTA) results with 11 respondents showed problems with usability and needed recommendations for interface improvements. The result of user experience evaluation is user interface recommendation to enhance the technician's performance to be more optimal. () © 2022 IEEE.

What do older adults think about when formulating implementation intentions for physical activity? Evidence from a qualitative study.

OBJECTIVES: Physical activity is an important health behaviour especially for older adults. Forming implementation intentions is an effective strategy to implement physical activity in daily life for young and middle-aged adults. However, evidence for older adults is inconclusive. This study explored the thoughts of older adults about implementation intentions and potential barriers and facilitators while formulating them. METHODS: Three samples of older adults from the United Kingdom (n = 8), Germany (n = 9) and Switzerland (n = 17) were prompted to think aloud while formulating implementation intentions to be more physically active. After the task, semi-structured interviews were conducted. Data were analysed thematically. RESULTS: Participants expressed pre-established thoughts about implementation intentions (e.g. they feel too restrictive). During the formulation of implementation intentions, several barriers to creating them were reported (e.g. problems with finding cues due to absence of recurring daily routines), but participants also mentioned that forming implementation intentions acted as a facilitator for physical activity (e.g. cues as useful reminders to be active, task itself triggering self-reflection about physical activity). After the task, participants reflected on circumstances that decrease the likelihood of enacting implementation intentions (e.g. spontaneous alternative activities, weather, health-related barriers, Covid-19-related barriers), which triggered spontaneous coping planning. CONCLUSIONS: The results on barriers and facilitators of implementation intentions and physical activity from older adults' perspectives provide starting points for improving instructions for older adults on how to create implementation intentions for physical activity. Future studies are needed to investigate whether the findings extend to implementation intentions for other behaviours.

Usability Testing of the Kidney Score Platform to Enhance Communication About Kidney Disease in Primary Care Settings: Qualitative Think-Aloud Study.

BACKGROUND: Patient awareness of chronic kidney disease (CKD) is low in part due to suboptimal testing for CKD among those at risk and lack of discussions about kidney disease between patients and clinicians. To bridge these gaps, the National Kidney Foundation developed the Kidney Score Platform, which is a web-based series of tools that includes resources for health care professionals as well as an interactive, dynamic patient-facing component that includes a brief questionnaire about risk factors for kidney disease, individualized assessment of risk for developing CKD, and self-management tools to manage one's kidney disease. OBJECTIVE: The aim of this study is to perform usability testing of the patient component of the Kidney Score platform among veterans with and at risk for kidney disease and among clinicians working as primary care providers in Veterans Affairs administration. METHODS: Think-aloud exercises were conducted, during which participants (veterans and clinicians) engaged with the platform while verbalizing their thoughts and making their perceptions, reasonings, and decision points explicit. A usability facilitator observed participants' behaviors and probed selectively to clarify their comprehension of the tool's instructions, content, and overall functionality. Thematic analysis on the audio-recording transcripts was performed, focusing on positive attributes, negative comments, and areas that required facilitator involvement. RESULTS: Veterans (N=18) were 78% (14/18) male with a mean age of 58.1 years. Two-thirds (12/18) were of non-White race/ethnicity, 28% (5/18) had laboratory evidence of CKD without a formal diagnosis, and 50% (9/18) carried a diagnosis of hypertension or diabetes. Clinicians (N=19) were 29% (5/17) male, 30% (5/17) of non-White race/ethnicity, and had a mean of 17 (range 4-32) years of experience. Veterans and clinicians easily navigated the online tool and appreciated the personalized results page as well as the inclusion of infographics to deliver key educational messages. Three major themes related to content and communication about risk for CKD emerged from the think-aloud exercises: (1) tension between lay and medical terminology when discussing kidney disease and diagnostic tests, (2) importance of linking general information to concrete self-management actions, and (3) usefulness of the tool as an adjunct to the office visit to prepare for patient-clinician communication. Importantly, these themes were consistent among interviews involving both veterans and clinicians. CONCLUSIONS: Veterans and clinicians both thought that the Kidney Score Platform would successfully promote communication and discussion about kidney disease in primary care settings. Tension between using medical terminology that is used regularly by clinicians versus lay terminology to promote CKD awareness was a key challenge, and knowledge of this can inform the development of future CKD educational materials.

A Pilot Study Conducting Online Think Aloud Qualitative Method during Social Distancing: Benefits and Challenges.

COVID-19 social distancing restrictions provided unprecedented insights into online research methodologies and approaches for both participants and researchers. Field research traditionally conducted face-to-face had to be transferred online, highlighting the great strides made in communication technologies (particularly live video streaming) over the last two decades for online qualitative research. However, dedicated research on these phenomena is tentative, including with regard to specific methods such as Think Aloud. This paper contributes to literature on online Think Aloud in qualitative research, evaluating new insights on its adoption online. It draws on findings from an online piloting study of Think Aloud tasks to explore the implications of using real-time internet video calls via SoIP applications by MS Teams. To assess the online Think Aloud process, this review called upon some of the comments made by participants during the semi-structured interview or comments made during the Think Aloud process, when they were relevant to the online process itself. It focuses on different dimensions of benefits, rapport in the session's encounter, challenges, and ethical concerns. Overall, the findings indicate that online Think Aloud sessions cannot completely replace in-person sessions for some particular and highly in-depth research areas, but they can greatly facilitate qualitative data collection in most conventional contexts. It is necessary to carry out further studies exploring the use of this and other online approaches and instructions.

Progressing the development of a food literacy questionnaire using cognitive interviews.

OBJECTIVE: Food literacy is the knowledge, skills and behaviours needed to meet food needs and determine intake and is conceptualised as eleven components under four domains of planning and managing, selecting, preparing, and eating. Previous measures of food literacy vary in their adherence to the conceptualisation and ability to capture totality of eating. This study aimed to determine items for inclusion and exclusion in a food literacy item pool and capture the general public's interpretation of everyday food literacy practices. DESIGN: Beginning with an item pool from previous studies, cognitive interviews were conducted using think-aloud and verbal probing methods. Data were first analysed for applicability, clarity, ambiguity and logic, then for emergent themes to ensure items captured the totality of the participant's eating. SETTING: Australia. PARTICIPANTS: Australian residents over 18 years of age recruited via Facebook residential groups (n 20). RESULTS: Of the original 116 items, 11 items had limited applicability; 13 items had unclear references; 32 items had lexical problems and 11 items had logical problems. In total, 29 items were deleted, 31 retained and 56 revised. Thematic analysis revealed participants limited their responses to consider only conventional practices such as grocery shopping, cooking and planned meals rather than the totality of their eating. An additional eighty-four items were developed to address eating out, incidental eating occasions and inconsistencies between participants assumed correct knowledge and that of public health guidelines. This resulted in a refined 171-item pool. CONCLUSIONS: This study progresses the development towards a comprehensive, validated food literacy questionnaire.

Assessment of Holographic Environment for Learning Sensing Technologies in CEM Education

Despite the benefits and suitability of mixed reality as a learning tool for spatial and haptic delivery of educational content, the adaptability of mixed reality for teaching sensing technologies in construction education remains ambiguous. Sensing technologies have potentials to advance construction performance as highlighted by several studies and construction industry reports. To equip future workforce with the required skills, it is important they experientially deploying these technologies on the jobsite. However, institutions are experiencing different constraints such as weather, schedule, and COVID-19 restrictions to provide future workforce with required hands-on learning. The study argues the pedagogical value of a mixed reality learning environment for teaching sensing technologies through a usability study with construction engineering and management students who are familiar with sensing technologies. The study employed quantitative and subjective users' evaluation through a cognitive walkthrough of the learning environment using a mixed reality head-mounted display. Subjective data were obtained using think-aloud protocol and questionnaires on users' experience. Eye tracking device inbuilt in the mixed reality head-mounted display was employed to monitor the eye gaze movements and provide quantitative data of users' cognitive workload. Results from eye tracking, think-aloud protocol and questionnaires were triangulated to validate findings from the study. The study reveals the ease of use and effectiveness of the environment for learning sensing technologies. The characteristics, benefits, and deficiencies of the learning environment as a pedagogical tool in construction education are further highlighted. © 2021 Computing in Civil Engineering 2021 - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2021. All rights reserved.

Investigating pharmacy students' therapeutic decision-making with respect to antimicrobial stewardship cases.

BACKGROUND: Antimicrobial stewardship programs (ASPs) play a big role in minimizing antimicrobial resistance. Pharmacists are essential members of the health care team and in order for them to fulfill roles on ASP teams and become antimicrobial stewards, they must be prepared adequately by pharmacy schools prior to entry into actual practice. Although programming has been implemented into entry-to-practice programs worldwide, little is known about how students interpret antimicrobial stewardship (AMS) data and arrive at clinical decisions. We aimed to explore students' cognitive processes and determine how they formulate therapeutic decisions when presented with AMS cases. METHODS: This was a qualitative study conducted using a case study approach, in which a sample (n=20) of pharmacy students was recruited to interpret AMS cases. Semi-structured 1-on-1 interviews were arranged with each participant. A think-aloud procedure with verbal protocol analysis was adopted to determine students' decision-making processes. Thematic analysis was used to interpret themes from the interview data. RESULTS: Two themes were interpreted from the data: students' focus and students' approach to case interpretation. Students' focus relates to external factors students consider when interpreting AMS case data and use to make and justify therapeutic decisions including patient-centered factors, drug-related factors, AMS interventions, and pharmacist's role. Students' clinical reasoning describes the approach that students use to interpret the data and the decision-making processes they employ to arrive at a clinical decision including a systematic approach versus non-systematic approach. CONCLUSIONS: Students vary in their focus and the cognitive strategies used to interpret AMS cases. Findings support the notion that clinical reasoning and decision-making should be explicitly taught in pharmacy curricula, in order to help students become aware of their own cognitive processes and decision-making abilities.

A Suggested Integrated Approach for the Evaluation of Remote Intranet Usability: A Case Study of Remote Workers During a Pandemic

As the societal impacts of COVID-19 progress, so do the workplace and business challenges the pandemic has brought on. It is now crucial to conduct user testing to measure and optimize the usability of corporate Intranets under pandemic conditions. This paper suggests and validates an integrated approach for intranet remote usability evaluation validated by user testing in a work-from-home context during a pandemic. The main contribution of the current research is the discovery of some user preferences specific to work-from-home workers during a pandemic not previously addressed in the literature on intranets.

User-Centered Design of A Novel Risk Prediction Behavior Change Tool Augmented With an Artificial Intelligence Engine (MyDiabetesIQ): A Sociotechnical Systems Approach.

BACKGROUND: Diabetes and its complications account for 10% of annual health care spending in the United Kingdom. Digital health care interventions (DHIs) can provide scalable care, fostering diabetes self-management and reducing the risk of complications. Tailorability (providing personalized interventions) and usability are key to DHI engagement/effectiveness. User-centered design of DHIs (aligning features to end users' needs) can generate more usable interventions, avoiding unintended consequences and improving user engagement. OBJECTIVE: MyDiabetesIQ (MDIQ) is an artificial intelligence engine intended to predict users' diabetes complications risk. It will underpin a user interface in which users will alter lifestyle parameters to see the impact on their future risks. MDIQ will link to an existing DHI, My Diabetes My Way (MDMW). We describe the user-centered design of the user interface of MDIQ as informed by human factors engineering. METHODS: Current users of MDMW were invited to take part in focus groups to gather their insights about users being shown their likelihood of developing diabetes-related complications and any risks they perceived from using MDIQ. Findings from focus groups informed the development of a prototype MDIQ interface, which was then user-tested through the "think aloud" method, in which users speak aloud about their thoughts/impressions while performing prescribed tasks. Focus group and think aloud transcripts were analyzed thematically, using a combination of inductive and deductive analysis. For think aloud data, a sociotechnical model was used as a framework for thematic analysis. RESULTS: Focus group participants (n=8) felt that some users could become anxious when shown their future complications risks. They highlighted the importance of easy navigation, jargon avoidance, and the use of positive/encouraging language. User testing of the prototype site through think aloud sessions (n=7) highlighted several usability issues. Issues included confusing visual cues and confusion over whether user-updated information fed back to health care teams. Some issues could be compounded for users with limited digital skills. Results from the focus groups and think aloud workshops were used in the development of a live MDIQ platform. CONCLUSIONS: Acting on the input of end users at each iterative stage of a digital tool's development can help to prioritize users throughout the design process, ensuring the alignment of DHI features with user needs. The use of the sociotechnical framework encouraged the consideration of interactions between different sociotechnical dimensions in finding solutions to issues, for example, avoiding the exclusion of users with limited digital skills. Based on user feedback, the tool could scaffold good goal setting, allowing users to balance their palatable future complications risk against acceptable lifestyle changes. Optimal control of diabetes relies heavily on self-management. Tools such as MDMW/ MDIQ can offer personalized support for self-management alongside access to users' electronic health records, potentially helping to delay or reduce long-term complications, thereby providing significant reductions in health care costs.

Think-Aloud Testing of a Novel Safer Drinking App for College Students During COVID-19: Usability Study.

BACKGROUND: Hazardous alcohol consumption, and binge drinking in particular, continues to be common among college students, posing the greatest risk for their health and safety. Despite widespread exposure to evidence-based preventive interventions among US undergraduates, only modest and temporary effects on risky drinking occur. Formative studies have demonstrated that students want a more engaging intervention tool for risky drinking that can be used just in time. OBJECTIVE: The purpose of this study is to test the appeal, relevance, and perceived utility of a draft mobile app for safer student drinking at a public university in Virginia. METHODS: Undergraduate student participants tested the draft mobile app via a web-based prototype that tailors to individual feedback with hot spots that responded to their taps to mimic app functionality. They narrated their impressions, navigation, and comments in a standardized think-aloud procedure. After each round of think-aloud interviews, researchers debriefed the investigators and developers to discuss findings and brainstorm app modifications. RESULTS: Minor changes to the functionality and aesthetics would improve usability of the app (eg, option for light mode in app settings). Student testers recommended tailoring the app to the needs of college students and to aspects of the local university's drinking culture. CONCLUSIONS: Findings from this study will be synthesized with information gained from other formative work to determine the final app features. We will test the app in a pilot randomized trial to assess app use and the impact of the app on college student drinking behavior over several months.

Development of and Experiences With an Informational Website on Early Labor: Qualitative User Involvement Study.

BACKGROUND: The period of regular contractions before 4 cm of cervical dilatation is often referred to as the latent phase or early labor. Women find it challenging to prepare for and cope with this phase of labor, and easily accessed web-based information from reliable sources may be useful in this preparation. OBJECTIVE: The aim of this study is to describe the development of a Norwegian website, Latens.no, for people seeking information on early labor and to explore users' experiences with the website to increase its user-friendliness. METHODS: We developed a website using an iterative process involving a multidisciplinary research team, health personnel, users, a graphic designer, and an expert in software development. We explored the website's user-friendliness using semistructured individual interviews and the think-aloud method. All interviews were audio recorded and transcribed. We then analyzed the participants' feedback on the website. RESULTS: Participants included women who had recently given birth to their first baby (n=2), women who were pregnant with their first baby (n=4), and their partners (n=2). Results from participants' experiences completing tasks included positive feedback related to the content of Latens.no, positive feedback related to the website's design, and suggestions for improvement. Participants wanted to find information on early labor on the internet. Moreover, they found the information on the website relevant, trustworthy, and easy to read, and the design was attractive and easy to use. Overall, the participants performed the tasks easily, with few clicks and minimal effort. CONCLUSIONS: The think-aloud method, while performing tasks, allowed for detailed feedback. The participants confirmed the user-friendliness of the website but at the same time provided information enabling improvement. We expect that changes made based on this user-centered design study will further increase the usability and acceptability of Latens.no.